Project for Production of Closed-Caption TV Programs for the Hearing Impaired

We describe an on-going project whose primary aim is to establish the technology of producing closed captions for TV news programs efficiently using natural language processing and speech recognition techniques for the benefit of the hearing impaired in Japan. The project is supported by the Telecommunications Advancement Organisation of Japan with the help of the ministry of Posts and Telecommunications. We propose natural language and speech processing techniques should be used for efficient closed caption production of TV programs. They enable us to summarise TV news texts into captions automatically, and synchronise TV news texts with speech and video automatically. Then the captions are superimposed on the screen. We propose a combination of shallow methods for the summarisation. For all the sentences in the original text, an importance measure is computed based on key words in the text to determine which sentences are important. If some parts of the sentences are judged unimportant, they are shortened or deleted. We also propose keyword pair model for the synchronisation between text and speech. Introduction The closed captions for TV programs are not provided widely in Japan. Only 10 percent of the TV programs are shown with captions, in contrast to 70 % in the United States and more than 30 % in Britain. Reasons why the availability is low are firstly the characters used in the Japanese language are complex and many. Secondly, at the moment, the closed captions are produced manually and it is a time-consuming and costly task. Thus we think the natural language and speech processing technology will be useful for the efficient production of TV programs with closed captions. The Telecommunications Advancement Organisation of Japan with the support of the ministry of Posts and Telecommunications has initiated a project in which an electronically available text of TV news programs is summarised and syncrhorinised with the speech and video automatically, then superimposed on the original programs. It is a five-year project which started in 1996, and its annual budget is about 200 million yen. In the following chapters we describe main research issues in detail and the project schedule, and the results of our preliminary research on the main research topics are presented.